What happened?

The space shuttle Challenger was ready to launch on the morning of Tuesday, January 28, 1986.

Figure 1. Prepared to launch

This was a symbiolic flight.

Figure 2. The crew

The night before the launch there was a 3 hour teleconference between Morton Thoikol, Marshall Space Flight Center, and Kennedy Space Center. The subject of the discussion was the sensitivity of O-rings, which seal the joints in booster rockets, to low temperatures.

Figure 3. O-rings seal joints on booster rockets

The discussion considered the data below.

Figure 4. O-ring failure as a function of temperature

Figure 4. O-ring failure as a function of temperature

  • Upon inspection of this plot, most of the participants concluded that, since the plot is “U” shaped, there was no evidence that temperature affected O-ring failure.

  • Lesson: the absence of evidence \(\neq\) evidence of absence.

In contrast, at least one participant, Roger Boisjoly, stated that “temperature was indeed a discriminator”.

Regardless, the Challenger was launched the following morning.

Figure 5. Challenger launch (NASA)

73 seconds into the flight, the shuttle exploded.

Figure 6. The O-rings failed

What should have happened?

Following the accident, President Regan appointed William Rogers to lead a committee to investigate the incident.

The committee concluded that “A combustion gas leak through the right Solid Rocket Motor aft field joint initiated at or shortly after ignition eventually weakened and/or penetrated the External Tank initiating vehicle structural breakup and loss of the Space Shuttle Challenger.”

Further, the committee concluded that “careful analysis of the flight history of the O-ring performance would have revealed the correlation of O-ring damage in low temperature”.

To understand the message from the committee, let’s look at the data:

Flight Temp Pressure O.ring Number
1 66 50 0 6
2 70 50 1 6
3 69 50 0 6
4 68 50 0 6
5 67 50 0 6
6 72 50 0 6
7 73 100 0 6
8 70 100 0 6
9 57 200 1 6
10 63 200 1 6
11 70 200 1 6
12 78 200 0 6
13 67 200 0 6
14 53 200 2 6
15 67 200 0 6
16 75 200 0 6
17 70 200 0 6
18 81 200 0 6
19 76 200 0 6
20 79 200 0 6
21 75 200 2 6
22 76 200 0 6
23 58 200 1 6
  • The number of O-rings that failed during 23 total flights was recorded with the temperature and pressure of the joints at launch.

Notice that some observations, those where 0 O-ring failures occured, were omitted.

Figure 7. O-ring failure as a function of temperature

Figure 7. O-ring failure as a function of temperature

  • Lessons:
    • ommission of outliers can be dangerous
    • outlier detection can be conditional on the inclusion/omission of other observations

O-ring failure might also be sensitive to pressure.

Figure 8. Number of O-ring failures as a function of temperature and pressure

  • Lesson: sometimes its valuable to consider several explanatory variables that may contribute to the response.

With the full dataset, another relationship(s) is apparent. To test the influence of temperature and pressure on O-ring failure, we have a couple options.

  • One option is to model the number of O-ring failures as a function of temperature and pressure.
    • Are there any problems with this approach?
  • Another option is to model O-ring failure as a binary function of temperature and pressure.
    • Are there any problems with this approach?
Dependent variable:
binomial binary
No.of O-ring failures O-ring failure
Temp -0.098** -0.229**
(0.045) (0.110)
Pressure 0.008 0.010
(0.008) (0.009)
Constant 2.520 13.292*
(3.487) (7.664)
Observations 23 23
Log Likelihood -15.053 -9.391
Akaike Inf. Crit. 36.106 24.782
Note: p<0.1; p<0.05; p<0.01

Consider the binomial model below:

\[ logit\left(\hat{\pi}\right)= 2.520 - 0.098\text{ temperature} + 0.008\text{ pressure} \] - And the inverse odds ratio:

##               Estimate Std. Error   z value  Pr(>|z|)
## (Intercept) 0.08044395 0.03059912 0.4853986 0.6251197
## Temp        1.10329014 0.95610229 8.9325907 0.9718580
## Pressure    0.99155187 0.99235230 0.3311752 0.7640570
  • For the binomial logistic regression model with temperature and pressure included, we see that a unit decrease in temperature changes the odds of failure by 1.1 times.

Now consider the binary model below:

\[ logit\left(\hat{\pi}\right)= 13.292 - 0.229\text{ temperature} + 0.010\text{ pressure} \] - And the inverse odds ratio:

##                 Estimate   Std. Error   z value  Pr(>|z|)
## (Intercept) 1.687336e-06 0.0004694408 0.1765067 0.9204913
## Temp        1.256928e+00 0.8958446419 7.9968563 0.9630857
## Pressure    9.896537e-01 0.9910609925 0.3140357 0.7813261
  • For the binary logistic regression model with Temperature and Pressure included, we see that a unit decrease in temperature changes the odds of failure by 1.25 times.

What did we miss?

Resources: